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1.4.1 Independence 

Let A be the event that it rains tomorrow, and suppose that P(A ) = y. Also suppose that I toss a fair coin; let 
B be the event that it lands heads up. We have P(B) = -j. 

Now I ask you, what is P(A | B)l What is your guess? You probably guessed that P(A\B) = P(A) = j. 
You are right! The result of my coin toss does not have anything to do with tomorrow's weather. Thus, no matter 
if B happens or not, the probability of A should not change. This is an example of two independent events. Two 
events are independent if one does not convey any information about the other. Let us now provide a formal 
definition of independence. 


Two events A and B are independent if P(A n B) = P(A)P(B). 


Now, let's first reconcile this definition with what we mentioned earlier, P(A\B) = P(A) . If two events are 
independent, then P(A D B ) = P(A)P(B), so 


P(A\B) 


P(AnB) 

P(B) 

PA)P(B) 

P(B) 

P{A). 


Thus, if two events A and B are independent and P(B ) 0, then P(A\B) = P(A). To summarize, we 

can say "independence means we can multiply the probabilities of events to obtain the probability of their 
intersection", or equivalently, "independence means that conditional probability of one event given another is the 
same as the original (prior) probability". 

Sometimes the independence of two events is quite clear because the two events seem not to have any physical 
interaction with each other (such as the two events discussed above). At other times, it is not as clear and we 
need to check if they satisfy the independence condition. Let's look at an example. 


Example 1.20 

I pick a random number from {1, 2, 3, • • • , 10}, and call it N. Suppose that all outcomes are equally likely. 
Let A be the event that N is less than 7, and let B be the event that N is an even number. Are A and B 
independent? 

• Solution 

o We have A = (1,2,3,4,5,6}, B = (2,4,6,8,10},andin5= (2,4,6}. Then 

P(A) = 0.6, 

P(B ) = 0.5, 


https://www.probabilitycourse.com/chapter1/1_4_1Jndependence.php 


1/6 




9/8/2018 


Independence | Conditional Independence 


P(A n B) = 0.3 

Therefore, P(A n B) = P(A)P(B) , so A and B are independent. This means that knowing 
that B has occurred does not change our belief about the probability of A. In this problem the two 
events are about the same random number, but they are still independent because they satisfy the 
definition. 


The definition of independence can be extended to the case of three or more events. 
Three events A , B , and C are independent if all of the following conditions hold 

P(A n B) = P(A)P(B), 

P{A nc) = P(A)P(C), 
P(BnC) = P(B)P(C), 

P{A nBnC) = P(A)P(B)P(C). 


Note that all four of the stated conditions must hold for three events to be independent. In particular, you can 
find situations in which three of them hold, but the fourth one does not. In general, for 71 events 
A \, A2 , • • •, A n to be independent we must have 

P(Ai fl Aj) = P(Ai)P(Aj), for all ij e {1,2, 

P(Ai n Aj n A k ) = P(Ai)P(Aj)P(A k ), for alH, j,k <E {1,2,- • • ,n}; 


P(A! nA 2 nA 3 ---nA n ) = p{a 1 )p{a 2 )p{a 3 )■ ■ ■ p{A n ). 

This might look like a difficult definition, blit we can usually argue that the events are independent in a much 
easier way. For example, we might be able to justify independence by looking at the way the random experiment 
is performed. A simple example of an independent event is when you toss a coin repeatedly. In such an 
experiment, the results of any subset of the coin tosses do not have any impact on the other ones. 


Example 1.21 

I toss a coin repeatedly until I observe the first tails at which point I stop. Let X be the total number of coin 
tosses. Find P[X = 5). 

• Solution 
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o Here, the outcome of the random experiment is a number X. The goal is to find -P(-A) = P{ 5). 
But what does X = 5 mean? It means that the first 4 coin tosses result in heads and the fifth one 
results in tails. Thus the problem is to find the probability of the sequence HHHHT when 
tossing a coin five times. Note that HHHHT is a shorthand for the event "(The first coin toss 
results in heads) and (The second coin toss results in heads) and (The third coin toss results in 
heads) and (The fourth coin toss results in heads) and (The fifth coin toss results in tails)." Since all 
the coin tosses are independent, we can write 

P(HHHHT) = P(H)P(H)P(H)P{H)P{T) 

= i_ i_ l l l 
2 ‘ 2 ‘ 2 ‘ 2 ‘ 2 
__ X 

32' 


Discussion: Some people find it more understandable if you look at the problem in the following way. I never 
stop tossing the coin. So the outcome of this experiment is always an infinite sequence of heads or tails. The 
value X (which we are interested in) is just a function of the beginning part of the sequence until you observe a 
tails. If you think about the problem this way, you should not worry about the stopping time. For this problem it 
might not make a big difference conceptually, but for some similar problems this way of thinking might be 
beneficial. 


We have seen that two events A and B are independent if P( A fl B) = P(A)P(B) . In the next two 
results, we examine what independence can tell us about other set operations such as compliments and unions. 

Lemma 1.1 

If A and B are independent then 

• A and B c are independent, 

• A c and B are independent, 

• A c and B c are independent. 

Proof 

We prove the first one as the others can be concluded from the first one immediately. We have 

P(AnB c ) = P(A-B) 

= P(A) - P(A n B) 

= P{A) — P(A)P(B) since A and B are independent 
= P(A)(1 - P(B)) ' 

= P(A)P(B C ). 

Thus, A and B c are independent. 
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Sometimes we are interested in the probability of the union of several independent events A\ , A 2 , • • • , A n . 

For independent events, we know how to find the probability of intersection easily, but not the union. It is 
helpful in these cases to use De Morgan's Law: 

A x u a 2 u • • • u A n = (A\ n a c 2 n • • • n A c n ) c 

Thus we can write 

p(Ai u^u-'-uiJ = 1 - p(a\ n a c 2 n • • • n A c n ) 

= 1 — P{A c 1 )P{A c 2 ) ■ • ■ P(An )(since the A^s are independent) 
= 1 - (1 - P(A 1 )) (1 - P(A 2 )) (1 - P(A n )). 


If Ai , A 2 , • • • , A n are independent then 



P(4iU4 2 U---U4„) = 1- ( 

1 - P(At)) (1 - P(A 2 )) ■ ■ ■ 

(l-P(O). 


Example 1.22 

Suppose that the probability of being killed in a single flight is p c = ^ ^ 6 based on available statistics. 

Assume that different flights are independent. If a businessman takes 20 flights per year, what is the probability 
that he is killed in a plane crash within the next 20 years? (Let's assume that he will not die because of another 
reason within the next 20 years.) 

• Solution 

o The total number of flights that he will take during the next 20 years is N = 20 x 20 = 400. 
Let p s be the probability that he survives a given single flight. Then we have 

Pa = 1 ~Pc- 

Since these flights are independent, the probability that he will survive all N = 400 flights is 
P(Survive N flights) = p s x p s x ■ ■ ■ x p s = p^ = (1 — p c ) N . 

Let A be the event that the businessman is killed in a plane crash within the next 20 years. Then 

P(A) = 1 - (1 -pc) N = 9.9995 x 1(T 5 « — !— . 
v ’ y yJ 10000 


Warning! One common mistake is to confuse independence and bein g disj oint . These are completely different 
concepts. When two events A and B are disjoint it means that if one of them occurs, the other one cannot occur, 
i.e., A fl B = 0. Thus, event A usually gives a lot of information about event B which means that they 
cannot be independent. Let's make it precise. 
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Lemma 1.2 

Consider two events A and B , with P(A) ^ 0 and P(Bj 0. If A and B are disjoint, then they are not 
independent. 

Proof 

Since A and B are disjoint, we have 

P(A Cl B) = 0 ± P(A)P(B). 

Thus, A and B are not independent. D 


Table 1.1 summarizes the two concepts of disjointness and independence. 


Concept 

Meaning 

Formulas 

Disjoint 

A and B cannot occur at the same time 

AnB = 0, 

P(A U B) = P(A) + P(B) 

Independent 

A does not give any information about B 

P(A\B) = P{A),P(B\A) = P(B ) 
P(A n B) = P(A)P(B) 


Table 1.1: Differences between disjointness and independence. 


Example 1.23 (A similar problem is given in [6]) 

Two basketball players play a game in which they alternately shoot a basketball at a hoop. The first one to make 
a basket wins the game. On each shot. Player 1 (the one who shoots first) has probability pi of success, while 
Player 2 has probability p 2 of success (assume 0 < pi , p 2 < 1). The shots are assumed to be independent. 

a. Find P(W\ ), the probability that Player 1 wins the game. 

b. For what values of pi and P 2 is this a fair game, i.e., each player has a 50 percent chance of winning the 
game? 

• Solution 

o In this game, the event W\ can happen in many different ways. We calculate the probability of each 
of these ways and then add them up to find the total probability of winning. In particular, Player 1 
may win on her first shot, or her second shot, and so on. Define A t as the event that Player 1 wins 
on her i'th shot. What is the probability of A r ? A, happens if Player 1 is unsuccessful at her first 
i — 1 shots and successful at her ith shot, while Player 2 is unsuccessful at her first i — 1 shots. 
Since different shots are independent, we obtain 

P(A 1 )=p 1 , 

P(M) = (1 ~l>l)(l -P2)Pl, 
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P{A 3 ) = (1 -Pl)(l -P 2 )(l “Pl)(l “P2)Pl, 

P{A k ) = [(1 -Pi)(l-p 2 )] fc Vi, 


Note that A \, A 2 , A 3 , • • • are disjoint events, because if one of them occurs the other one cannot 
occur. The event that Player 1 wins is the union of the A % 's, and since the A, 's are disjoint, we have 


P{Wi) = P(Ai u A 2 u As u • • •) 

= P(Ai) + P(A 2 ) + P(A^) + • • • 

= Pi + (1 -Pi)(l P 2 )pi + [(1 -Pi)(l -P2)] 2 Pi 


= Pi 


1 + (1 -pi)(l - P 2 ) + [(1 -Pl)(l -P 2 )] 


Note that since 0 < pi ,p 2 < 1, for X = (1 — Pl)(l — P 2 ) we have 0 < X < 1. Thus, 
using the geometric sum formula (^^_q CLxA = O,i'°r |,x < 1 ), we obtain 


P(W 1 ) = 


1 - (1 -Pl)(l -p 2 ) P 1 +P 2 -P 1 P 2 ' 


It is always a good idea to look at limit cases to check our answer. For example, if we plug in 
Pl = 0,p 2 7^ 0, we obtain P{W\ ) = 0, which is what we expect. Similarly, if we let 
P2 = 0,pi ^ 0, we obtain P{W\ ) = 1 , which again makes sense. 

Now, to make this a fair game (in the sense that P{W\ ) = .5), we have 

P(VFi) = -—-= 0.5 

P 1 +P 2 - P 1 P 2 


and we obtain 


Pl 


P2 

1 +P2 ' 


Note that this means that pi < p 2 , which makes sense intuitively. Since Player 1 has the 
advantage of starting the game, she should have a smaller success rate so that the whole game is fair. 


previous 
next —* 
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